Accuracy adaptive symbology for head worn display

ABSTRACT

A system and method for modifying symbology displayed on a head worn display (HWD) based on a quality-of-service value is disclosed. The system includes a head worn display configured to display symbology, a tracking camera, a controller, and a tracking processing unit configured to receive inputs form the tracking camera and transmit a signal quality value to the controller. The controller includes processors and memory with instructions that when executed by the processors, cause the processors to display the symbology on the display, determine a quality-of-service value based on at least the signal quality value, and modify at least one symbol of the symbology based on the quality-of-service value from a default symbol to a degraded symbol if the quality-of-service value changes from a high-quality value to a low-quality value. The system may also receive sensor signals from other sensors, and use the sensor signal to determine the quality-of-service value.

BACKGROUND

In a head worn display (HWD) the accuracy of the head tracking solutioncan vary over time depending on a variety of factors: operatingenvironment, health of equipment, geometry, installation constraints,aircraft and/or head motion histories, sensor errors and latencies.These variations can cause one or more symbols of a symbology displayedon the HWD to be less accurate or precise. Imprecise/inaccuratesymbology can lead to hazardous situations; however, animprecise/inaccurate symbol of a symbology can still provide value to apilot as long as the pilot has information relating to the accuracyand/or precision of the symbol. Currently, HWD systems are unable toconvey the accuracy or the precision of displayed symbology to a uservia the HWD. Therefore, it is desirable to provide a system or methodthat can determine the accuracy and/or precision of a symbol of an HWDsymbology, and convey that accuracy and/or precision information to auser.

SUMMARY

A system is disclosed. In one or more embodiments, the system includes ahead worn display configured to display symbology of an operatingenvironment. In one or more embodiments, the system further includes atracking camera. In one or more embodiments, the system further includesa tracking processing unit communicatively coupled to the trackingcamera. In one or more embodiments, the tracking processing unit isconfigured to determine a position and orientation of the head worndisplay. In one or more embodiments, the tracking processing unit isconfigured to transmit a position and orientation signal to acontroller, wherein the position and orientation signal includes asignal quality value. In one or more embodiments, the system furtherincludes a controller communicatively coupled to the head worn displayand the tracking camera. In one or more embodiments, the system furtherincludes one or more processors and a memory with instructions storedupon. In one or more embodiments, the instructions, when executed by theone or more processors, cause the one or more processors to display thesymbology on the head worn display. In one or more embodiments, theinstructions, when executed by the one or more processors, further causethe one or more processors to determine a quality-of-service value basedon at least the signal quality value. In one or more embodiments, theinstructions, when executed by the one or more processors, further causethe one or more processors to modify at least one symbol of thesymbology based on the quality-of-service value from a default symbol toa degraded symbol if the quality-of-service value changes from ahigh-quality value to a low-quality value.

In some embodiments of the system, the instructions further cause theone or more processors to receive a sensor signal from one or moresensors and use the sensor signal to determine the quality-of-servicevalue.

In some embodiments of the system, the quality-of-service valuecomprises an accuracy characteristic.

In some embodiments of the system the quality-of-service value comprisesa precision characteristic.

In some embodiments of the system the quality-of-service value comprisesa signal strength characteristic.

In some embodiments of the system the degraded symbol is configured as aremoved symbol.

In some embodiments of the system the degraded symbol is configured witha different color than the default symbol.

In some embodiments of the system the degraded symbol is configured witha different line width or different from than the default symbol.

In some embodiments of the system the degraded symbol is configured witha different line type than the default symbol.

In some embodiments of the system wherein the controller iscommunicatively coupled to a vision system.

In some embodiments of the system, the system further comprises aninertial sensor communicatively coupled to at least one of the head worndisplay or the tracking processing unit.

In some embodiments of the system, the symbol represents a confidenceline of a path.

In some embodiments of the system, the path is configured as a runway.

A method for modifying symbology displayed on a head worn display (HWD)based on a quality-of-service value is disclosed. In one or moreembodiments, the method includes determining a position and orientationof the HWD. In one or more embodiments, the method further includestransmitting a position and orientation signal to a controller, whereinthe position and orientation signal includes a signal quality value. Inone or more embodiments, the method further includes determining aquality-of-service value based on at least the signal quality value. Inone or more embodiments, the method further includes displayingsymbology on the HWD, wherein at least one symbol of the symbology ismodified from a default form to a degraded form based on thequality-of-service value.

In one or more embodiments of the method, the method further includesreceiving a sensor signal from one or more sensors. In one or moreembodiments of the method, the method further includes using the sensorsignal to determine the quality-of-service value.

This Summary is provided solely as an introduction to subject matterthat is fully described in the Detailed Description and Drawings. TheSummary should not be considered to describe essential features nor beused to determine the scope of the Claims. Moreover, it is to beunderstood that both the foregoing Summary and the following DetailedDescription are example and explanatory only and are not necessarilyrestrictive of the subject matter claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. The use of the same reference numbers in different instances inthe description and the figures may indicate similar or identical items.Various embodiments or examples (“examples”) of the present disclosureare disclosed in the following detailed description and the accompanyingdrawings. The drawings are not necessarily to scale. In general,operations of disclosed processes may be performed in an arbitraryorder, unless otherwise provided in the claims. In the drawings:

FIG. 1 is a diagram of a HWD system, in accordance with one or moreembodiments of the disclosure;

FIG. 2 is a block diagram of the componentry of the HWD system, inaccordance with one or more embodiments of the disclosure;

FIG. 3A are drawings illustrating snapshots of an HWD display, inaccordance with one or more embodiments of the disclosure;

FIG. 3B are drawings illustrating snapshots of an HWD display, inaccordance with one or more embodiments of the disclosure;

FIG. 3C is a drawing illustrating a snapshot of an HWD display, inaccordance with one or more embodiments of the disclosure; and

FIG. 4 is a flowchart illustrating a method for modifying symbologydisplayed on a HWD based on a quality-of-service value, in accordancewith one or more embodiments of the disclosure.

DETAILED DESCRIPTION

Before explaining one or more embodiments of the disclosure in detail,it is to be understood that the embodiments are not limited in theirapplication to the details of construction and the arrangement of thecomponents or steps or methodologies set forth in the followingdescription or illustrated in the drawings. In the following detaileddescription of embodiments, numerous specific details may be set forthin order to provide a more thorough understanding of the disclosure.However, it will be apparent to one of ordinary skill in the art havingthe benefit of the instant disclosure that the embodiments disclosedherein may be practiced without some of these specific details. In otherinstances, well-known features may not be described in detail to avoidunnecessarily complicating the instant disclosure.

As used herein a letter following a reference numeral is intended toreference an embodiment of the feature or element that may be similar,but not necessarily identical, to a previously described element orfeature bearing the same reference numeral (e.g., 1, 1 a, 1 b). Suchshorthand notations are used for purposes of convenience only and shouldnot be construed to limit the disclosure in any way unless expresslystated to the contrary.

Further, unless expressly stated to the contrary, “or” refers to aninclusive or and not to an exclusive or. For example, a condition A or Bis satisfied by anyone of the following: A is true (or present) and B isfalse (or not present), A is false (or not present) and B is true (orpresent), and both A and B are true (or present).

In addition, use of “a” or “an” may be employed to describe elements andcomponents of embodiments disclosed herein. This is done merely forconvenience and “a” and “an” are intended to include “one” or “at leastone,” and the singular also includes the plural unless it is obviousthat it is meant otherwise.

Finally, as used herein any reference to “one embodiment” or “someembodiments” means that a particular element, feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment disclosed herein. The appearances of thephrase “in some embodiments” in various places in the specification arenot necessarily all referring to the same embodiment, and embodimentsmay include one or more of the features expressly described orinherently present herein, or any combination of sub-combination of twoor more such features, along with any other features which may notnecessarily be expressly described or inherently present in the instantdisclosure.

Disclosed is a system and method for presenting accuracy adaptivesymbology to a head worn display (HWD). In general, HWDs requiretracking devices that track the movement of the head of a user. Thistracking allows HWD systems to correctly display symbology onto theeyepieces of the HWD. Tracking devices require constant signalingbetween HWD components for correct displaying of symbology. Thissignaling can become suboptimal due to component failure, environmentalfactors (e.g., jamming) and other system malfunctions. Suboptimalsignaling can be catastrophic if the user is heavily dependent upon theHWD for hazardous tasks, such as landing an aircraft in low visibilityconditions. Within the disclosed system, a quality characteristic ofsignal between the head tracker and HWD components is determined, andthe symbology displayed on the eyepieces of the HWD are altered based onthe quality characteristic. The modified symbology quickly communicatesto the user a confidence value on the accuracy of the symbology based onthe tracker signaling without distracting or misleading the user.

FIG. 1 is a diagram of a HWD system 100, in accordance with one or moreembodiments of the disclosure. The HWD system 100 includes a HWD 104.The HWD 104 may be of any size or type, and includes any head mounteddisplay (HMD), head-mounted projected displays (HMPD) and retinalscanning displays (RSD), The HWD 104 includes one or two eyepieces 108that receive and display symbology to the user. The eyepieces 108 may beof any type of display technology including organic light emittingdisplays (OLED), light emitting diode (LED) displays, liquid crystaldisplays (LCD), liquid crystal on silicon (LCoS) displays, and minicathode ray tube (mini-CRT) displays.

The eyepieces 108 may be configured as spectacles/goggles, or may beincorporated into a helmet 112, hat or other head-worn equipment. TheHWD system 100 further includes a camera 116 (e.g., a head trackingcamera) coupled to the helmet 112, the eyepieces 108, or other aspectsof the HMD system 100. The HWD system 100 may further include fiducials120 arranged within the environment of the of the HWD system 100 (e.g.,located on an interior surface of a cockpit) The fiducials 120 arerecognizable by the HWD system 100 via the camera 116, and HWD systemcomponentry, which enable the HWD system 100 to determine the positionand orientation of the eyepieces 108 based on the angle and/or distancesbetween each fiducial 120 and the camera. The fiducials 120 may be ofany number (e.g., three to twenty), of any shape, and of any type. Forexample, the fiducials 120 may be configured as a set of four squarebar-codes or QR codes.

The HWD system 100 further includes a tracker processing unit 124configured to receive and process data from the camera (e.g., opticalposition/orientation data). In some embodiments, the tracker processingunit 124 is also configured to receive and process data from an inertial(acceleration and/or gyroscopic), or magnetic sensor.

FIG. 2 is a block diagram of the componentry of the HWD system 100, inaccordance with one or more embodiments of the disclosure. The HWDsystem 100 may include a capacity for multiple users at one time (e.g.,from 1 to 100 users). For example, the HWD system 100 may include a HWD104, a tracking processing unit 124, and a HWD control panel 204 forboth a pilot seat 208 and a copilot seat 212 (e.g., not shown forclarity). The copilot seat 212 may have one or more, or all, componentsof the pilot seat 208 and vice-versa. The control panel 204 isconfigured to allow a user to modify operation aspects of the HWD 104including but not limited to brightness, and the ability to turn on oroff a HWD mode, where turning off the HWD would remove any symbologyfrom the eyepieces 108. It should be noted that the one or more users ofthe HWD system 100 may not be pilots or copilots, and that the HWDsystem 100 may be used for purposes other than operating a vehicle.Therefore, the above language is intended to provide an illustration ofan embodiment of the HWD system 100, and not a limitation.

The HWD 104 may further include, or be communicatively coupled to, aninertial sensor 216 configured to collect inertial data that is sent tothe tracker processing unit 124. For example, the HWD system 100 may beconfigured to operate as a hybrid optical/inertial tracking HWD system,with the inertial data generated via the inertial sensor 216. The HWD104 may further include a light sensor 220 configured to determinelevels of ambient light in and around the HWD 104. For example, thelight sensor 220 may determine the amount of ambient light in a cockpit.

In embodiments, the HWD system 100 further includes a display computer224 communicatively coupled to the tracker processing unit 124. Thedisplay computer 224 receives position and orientation information fromthe tracker processing unit 124, and renders images, such as symbology,based on the position and orientation information before the image isviewed on the eyepieces 108. The display computer 224 may also modifythe symbology based on a quality of signal received from the trackerprocessing unit 124, the inertial sensor 216, the camera 116, the HWD104, or other componentry.

The HWD system 100 may include, or is communicatively coupled to, sensorand data collection componentry that provide the display computer 224with information that the display computer 224 translates into asymbology that is imaged onto the HWD 104. For example, the HWD system100 may include, or is communicatively coupled to, one or more sensorinput/outputs 228 a-b that communicate sensor readings (e.g., sensorsignals) to the HWD system 100, including but not limited to airspeedreadings, altitude readings, position readings (e.g., GPS positionreadings), and fuel level readings. In another example, the HWD system100 may include, or is communicatively coupled to, a vision system 232that provides an image of the surrounding environment to the displaycomputer 224. The vision system 232 may be configured as any type ofimage collecting system including but not limited to an enhanced visionsystem (EVS), an enhanced flight vision system (EFVS), a syntheticvision system (SVS), a combined vision system (CVS) a primary flightdisplay (PFD)).

Components of the HWD system 100 as well as componentry communicativelycoupled to the HWD system 100 rely on a digital information transfersystems (e.g., buses) for data transfer between componentry. The HWDsystem 100 may use any type of bus componentry or data transfer protocolincluding but not limited to an ARINC 429 data transferstandard/protocol and a ARINC 818 video interface and protocol standard.The HWD system 100 may use specialized data transfer links betweensystem componentry, such as a HWD crosstalk bus 236 that providescommunication between the HWD componentry of the pilot seat 208 with theHWD componentry of the Co-pilot seat 212. Components of the HWD system100 may utilize parallel and/or serial busses for communication withother componentry, and one or more components may include power inputs(e.g., AC power or DC power). One or more components of the HWD system100 may contain a controller 240 configured to provide processingfunctionality for the HWD system 100, including generating and/ormodifying symbology for the HWD 104. As shown in FIG. 2 , the displaycomputer 224 includes the controller 240, which comprises one or moreprocessors 244, a memory 248, and a computer interface 252. Othercomponents of the HWD system 100, or componentry communicatively coupledto the HWD system 100 may also utilize controllers to perform processingfunctionality for the HWD system 100 including but not limited to thetracker processing unit 124, the HWD 104, the vision system 232, thesensor input/output 228 a-b, the inertial sensor 216, and the camera116.

The one or more processors 244 may include any processor or processingelement known in the art. For the purposes of the present disclosure,the term “processor” or “processing element” may be broadly defined toencompass any device having one or more processing or logic elements(e.g., one or more micro-processor devices, one or more applicationspecific integrated circuit (ASIC) devices, one or more fieldprogrammable gate arrays (FPGAs), or one or more digital signalprocessors (DSPs)). In this sense, the one or more processors 244 mayinclude any device configured to execute algorithms and/or instructions(e.g., program instructions stored in memory 248). In one embodiment,the one or more processors 244 may be embodied as a desktop computer, aflight computer, mainframe computer system, workstation, image computer,parallel processor, networked computer, or any other computer systemconfigured to execute a program configured to operate or operate inconjunction with the HWD system 100, as described throughout the presentdisclosure. Moreover, different subsystems of the system 100 may includea processor or logic elements suitable for carrying out at least aportion of the steps described in the present disclosure. Therefore, theabove description should not be interpreted as a limitation on theembodiments of the present disclosure but merely as an illustration.

The memory 248 can be an example of tangible, computer-readable storagemedium that provides storage functionality to store various data and/orprogram code associated with operation of the controller 240 and/orother components of the HWD system 100, such as software programs and/orcode segments, or other data to instruct the controller 240 and/or othercomponents to perform the functionality described herein. Thus, thememory 248 can store data, such as a program of instructions foroperating the HWD system 100 or other components. It should be notedthat while a single memory 248 is described, a wide variety of types andcombinations of memory 248 (e.g., tangible, non-transitory memory) canbe employed. The memory 248 can be integral with the controller, cancomprise stand-alone memory, or can be a combination of both. Someexamples of the memory 248 can include removable and non-removablememory components, such as a programmable logic device, random-accessmemory (RAM), read-only memory (ROM), flash memory (e.g., a securedigital (SD) memory card, a mini-SD memory card, and/or a micro-SDmemory card), solid-state drive (SSD) memory, magnetic memory, opticalmemory, universal serial bus (USB) memory devices, hard disk memory,external memory, and so forth.

The communication interface 252 can be operatively configured tocommunicate with components of the controller 240 and other componentsof the HWD system 100. For example, the communication interface 252 canbe configured to retrieve data from the controller 240 or othercomponents, transmit data for storage in the memory 248, retrieve datafrom storage in the memory 248, and so forth. The communicationinterface 252 can also be communicatively coupled with controller 240and/or system elements to facilitate data transfer between systemcomponents.

The display computer 224 and the tracking processing unit 124 worktogether to provide an accurate rendering of symbology to the display.For example, once the display computer 224 has generated symbology data(e.g., based on inputs from the sensor input/output 228 a-b and/orvision system 232), the display computer 224 then receives tracking datafrom the tracking processing unit 124 and modifies the symbologyaccordingly. For instance, the tracking processing unit 124 may sendposition and orientation data that the tracking processing unit 124 hasitself processed from inputs from the tracker camera 116, the inertialsensor 216, the light sensor 220, the HWD 104, and/or the control panel204. The position and orientation data may include pose data, posealgorithms, command and control data, and other inputs that facilitateaccurate rendering by the display computer 224.

In embodiments, the HWD system 100 is configured to determine if aposition and orientation signal (e.g., containing the position andsignal data) is a quality signal (e.g., having a quality signal value).Data signaling can be disrupted at several points within the HWD system100. For example, one or more sensors (e.g., the camera 116, theinertial sensor, or the light sensor), may become compromised, orotherwise not perform competently. For instance, the one or more sensorsmay have an internal malfunction, such as a short circuit. In anotherinstance, the componentry of the one or more sensors (e.g., the camera116) may be affected by an environmental issue, such as smoke in acockpit. In another example, data may be disrupted at one or more databusses that connect components of the HWD system 100. One or moreprocessors from the display computer 224, the tracker processor unit124, and/or other components may determine whether one or more datasignals sent to and/or received by the display computer 224 and/or thetracker processing unit 124 are high-quality data signals or low-qualitydata signals. The display computer 224 receives these inputs (e.g., assignal quality values) and generates an overall quality-of-service (QoS)value, which provides both an estimate on the accuracy and/or precisionof the data, or specific data components (e.g., attitude data) receivedby the display computer 224, and assists in determining whatmodifications, if any, need to be made to the symbology generated by thedisplay computer 224 that communicate to the user the accuracy/precisionof one or more symbols of the HWD symbology in real time.

The HUD system 100 is configured to display symbology on the HWD 104that is modified based on the predicted accuracy of the symbology (e.g.,QoS value). Modifications to any symbol of symbology may be made uponany accuracy prediction. For example, the for HWD system 100 operatingwith high-accuracy and/or high QoS values (e.g., indicating a 99.9% ofaccuracy or high precision), a symbol within the symbology may appear asa default symbol, or as a symbol modified to represent high accuracy.Conversely, for a HWD system 100 operating with low-accuracy and/or lowQoS values (e.g., indicating less than 95% accuracy or low precision), asymbol may appear as a degraded symbol modified to represent lowaccuracy or may be removed entirely. Other modifications between adefault symbol, a degraded symbol, and/or a high-accuracy/precisionsymbol may include changes in color, and changed in line type (e.g.,dotted line versus solid line).

Examples of the symbology from HWD systems 100 operating under high orlow QoS values are shown in FIG. 3A, which illustrate snapshots ofdisplayed HWD images 300 a-d derived from the vision system 232, eachincluding a runway 304 a-d (e.g., a path), in accordance with one ormore embodiments of the disclosure. The displayed HWD images 300 a-demulate what a pilot would see when attempting to land and aircraft viathe HWD 104 under different QoS value conditions (e.g., 304 a under highQoS, 304 b-d, under low QoS). These different QoS value conditions arebased on the accuracy/precision of incoming data from the trackerprocessing unit 124 and indicate to the pilot the relative accuracy ofthe runway in an Earth-centered/Earth-fixed coordinate (ECEF) frame. Forexample, in displayed HWD image 300 a, a high-confidence line 308 a isshown that mimics the boundary of the runway 304 a. This singularhigh-confidence line 308 a may indicate that the HWD system 100 isworking under high QoS value conditions, and that the display as shownis highly correlative (e.g., accurate and/or precise) to the actualenvironment. For example, the high-confidence line 308 a may indicate tothe pilot that the midline 312 of the runway 304 as shown is a predictedto be within 95% to be within one meter of the actual midline of theactual runway.

Symbology modifications based on QoS value conditions may be applied toa single symbol within the symbology displayed on the display, a set ofsymbols within the symbology displayed on the display, or all symbologydisplayed on the display. For example, changes in a QoS value may resultin all earth-referenced/conformal signals being modified. For instance,and in reference to FIG. 3A. all earth-referenced signals, including theoutline/boundary of the runway 304, may be altered according to the QoSvalue. Therefore, the above description should not be considered alimitation of the HWD system 100, but as an illustration.

The modification of a symbol within the symbology to show high or lowconfidence may be presented in any number of ways. For example, thedisplayed HWD image 300 b may show a differentially-coloredlow-confidence line 310 b (e.g., indicated by the thickened line),indicating a low QoS condition (e.g., poor signals received from thetracker processing unit 124, the camera 116, or the inertial sensor216). In another example, the displayed HWD image 300 c may present thelow-confidence line 310 c in addition to the default high-confidenceline 308 c. For instance, the distance between the low-confidence line312 c and the high confidence line 308 c may give an indication of theconfidence level that the displayed runway 304 is presented correctly(e.g., the greater distance indicating a decreased QoS condition). Inanother example, the HWD image 300 d, may present the low-confidenceline 312 d as a dotted line.

In some embodiments, the QoS value comprises an accuracy characteristic,for which a symbol is modified based on the accuracy characteristic anddisplayed on the HWD 104. For example, the symbol may be displayed as adegraded or missing signal. For instance, and as shown above, upon adetermination that the QoS value has a low accuracy characteristic(e.g., less than 95% accurate), the high-confidence line 308 may beremoved. The symbol may also be modified based on a precisioncharacteristic, for which a symbol is modified based on the accuracycharacteristic and displayed on the HWD 104. For example, and as shownabove, upon a determination that the QoS value has a low precisioncharacteristic (e.g., less than one meter of precision), thehigh-confidence line 308 may be replaced with a low-precision line 310(e.g., a degraded symbol).

In some embodiments, the symbology with the HWD system 100 is configuredto change quantitatively based on the QoS value. For example, thehigh-confidence line 308 a may transform from a solid line to a highlybroken/dotted line increase depending on the level of accuracy of thesymbol as determined by the QoS value. For instance, a symbol with apredicted accuracy of 99% may be configured as a solid high-confidenceline 308 a, whereas a symbol with a predicted accuracy of 95% may beconfigured as a dashed high-confidence line 308 a (e.g., 2 dashes percm), and a symbol with a predicted accuracy of 90% may be configured asa dotted high confidence line 308 a (e.g., 4 dots per cm). The symbologymay utilize any type of transformation of a symbol to represent a changein QoS.

In another example, a confidence line 308 may be assigned a specificcolor based upon the predicted accuracy of the symbol (e.g., red for apredicted accuracy of 95%, and green for a predicted accuracy of 99%).

In some embodiments, the QoS value is based on accuracy/precisioncharacteristics of data received by the sensor input/output 228 a-band/or vision systems 232, or may be based on accuracy/precisioncharacteristics of data received by both the sensor input/output 228 a-band/or vision systems 232, as well as the tracking inputs received fromthe tracking processing unit 124 and related componentry (e.g., theinertial sensor 216, the camera 116, the light sensor 220, and/or theHWD 104.) For example, an attitude sensor that is performing slightlyout of normal operating parameters may not necessarily result in a lowQoS value. However, the same attitude sensor that reports to a HWDsystem 100 with a camera 116 that is slightly performing out of normaloperating parameters may additively result in a low QoS value condition,resulting in modified attitude symbology, such as that shown in FIG. 3B.Here, displayed HWD images 300 e-f indicate a displayed HWD image 300 eunder high QoS value conditions and a displayed HWD image 300 f underlow QoS value conditions, the low QoS value cause by both the attitudesensor and the camera 116 performing slightly out of normal operatingparameters. The symbology associated by the attitude sensor, the horizonpitch 316 and the pitch ladder 320 are removed from the HWD 104,indicating that the attitude measurements are not to be relied on.However, because the other sensors are working properly, and do notcause low QoS value conditions even with the camera 116 operating out ofnormal operating parameters, the corresponding symbols, such as theflight path vector 322, does not change.

In some embodiments, the QoS value is based on a signal strengthcharacteristic. For example, and as shown FIG. 3B, the low QoS value maybe due to a loss of signal strength from either the attitude sensor orthe camera 116.

In some embodiments, a low QoS condition may result in the appearance ofone or more low QoS icons 324 (e.g., the “Low Q” square in FIG. 3C)appearing on the HWD image 300 g. For example, the low QoS icon 324 mayappear when the HWD orientation is not to be trusted. The low QoS icon324 may be head-referenced to ensure that the symbol is always visibleregardless of the head orientation or the loss of competence in the headtracker or tracker processing unit 124 (e.g., the low QoS icon 324 isinternally consistent and referenced to the head). The low QoS icon 324may be configured as any shape, size, or type of signal. For example,the low QoS icon 324 may have an appearance similar to a standbyinstrument symbol or an unusual altitude symbol.

FIG. 4 is a flowchart illustrating a method 400 for modifying symbologydisplayed on a HWD based on a quality-of-service value, in accordancewith one or more embodiments of the disclosure. The method 400 may beused by any HWD system 100 including but not limited to EVS-based HWDsystems 100 for aircraft.

In some embodiments, the method includes a step 404 of determining aposition and orientation of the HWD 104. For example, the trackingprocessing unit 124 may determine the position and orientation of theHWD 104, components of the HWD (e.g., eyepieces 108, and/or thehead/eyes of the pilot, based on the position of HWD 104 and/or HWDcomponents.

In some embodiments, the method 400 further includes a step 408 oftransmitting a position and orientation signal to the controller 240,wherein the position and orientation signal includes a signal qualityvalue. For example, the tracking processing unit 124 may transmit aposition and orientation signal to the display computer 224 thatincludes not only the calculated position and orientation of the HWD104, but also a signal quality value that indicates how precise, and/orstrong the signal is.

In some embodiments, the method 400 further includes a step 412 ofdetermining a quality-of-service value based on at least the signalquality value. For example, the display computer, having received aquality signal value from the tracker processing unit 124 may assign aQoS value based on the quality signal value. The QoS value may alsoinclude inputs from other components of the HWD system 100 andcommunicatively coupled componentry.

In some embodiments, the system includes a step 416 of displayingsymbology on the HWD, wherein at least one symbol of the symbology ismodified from a default form to a degraded form based on the QoS value.For example, and as shown in displayed HWD images 300 a,d of FIG. 3A, adefault form of a symbol, the solid high-confidence line 308, may bemodified to a degraded form such as the dotted low-confidence line 314based on a determined QoS value.

In some embodiments, the method 400 further includes steps that involvesensor signals from sensors that include but are not limited to thesensor input/output 228 a-b and vision systems 232. For example, in someembodiments, the method 400 further includes a step 420 of receiving asensor signal from one or more sensors. The method may further include astep 424 of using the sensor signal to determine the quality-of-servicevalue. For example, and as shown in FIG. 3B, performance issues in theattitude sensor and the camera 116 may contribute to a low QoS valuethat leads to specific symbols of the attitude sensor, the horizon pitch316 and the pitch ladder 320 being removed from the symbology, whereasperformance issues in the attitude sensor and the camera 116, whenconsidered individually, would not lead to a change in symbology. Incases where the symbol is removed, another symbol may replace theremoved symbol. For example, in the case where the horizon pitch 316 andthe pitch ladder 320 symbols are removed, the symbols may be replaced bya simple heading compass.

The HWD system 100 and method 400 may be used with any type ofsymbology. For example, the HWD system 100 and method 400 may be used tomodify aircraft-related symbology for HWD systems used on board anaircraft. For instance, the aircraft-relate symbology used may includebut not be limited to command heading marker symbology, true headingindicator symbology, heading symbology, rate of climb/descent symbology,altitude symbology, barometric setting symbology, waterline symbology,course line symbology, horizon symbology, extended horizon barsymbology, elevation deviation symbology, energy symbology (e.g., anenergy caret), azimuth deviation symbology, gear up/down symbology, ILSsteering symbology, course line steering symbology, flight pathsymbology, pitch ladder symbology, AOA bracket symbology, velocityvector symbology, bank angle symbology, ghost velocity vector symbology,peak aircraft g-force symbology, aircraft g-force symbology, Mach numbersymbology, angle of attach symbology, airspeed symbology, landing zonesymbology, great circle steering symbology, as well as symbologyrepresenting any object detected by the vision system 232 (e.g., a pathor runway 304. It should be understood that symbology may not be limitedto shapes. For example, the symbology may include, or only contain,text.

In another example, the symbology modified within the method 400 and HWDsystem 100 may include a flight path circle, wherein the circle size isincreased as an accuracy of the HWD signaling decreased to demonstratethe angular accuracy of a presentation of a flight path. The flight pathcircle could also be changed to a dashed or dotted line representation.

In some embodiments, the HWD system 100 and/or method 400 may beconfigured to remove or modify checklist action prompts if the trackingenvironment is degraded. For example, buttons appearing on the HWDprompting the use to check aircraft components (e.g., check statuslight, or check oxygen bottle) may appear in a degraded state (e.g.,having a dotted outline) upon the HWD system 100 operating in a degradedtracking environment, which communicates to the user the degradedtracking status.

As demonstrated in FIG. 3A, any reference point that has symbologyrendered at its location (e.g., the runway 304) could utilize size,shape, and color to indicate relative accuracy in an ECEF frame.Uncertainty for such point features may also be represented by moretraditional error ellipses when appropriate. For example, and as shownin FIG. 3A, the width of lines, such as runway lines, could be increasedor otherwise modified to indicate the area where the runway is expectedto be located. The runway lines could also be removed entirely if theaccuracy of the head tracking solution is too low. In some cases,symbology color can be modified to reflect “binary for credit” checksagainst FAA regulations.

It is to be understood that embodiments of the methods disclosed hereinmay include one or more of the steps described herein. Further, suchsteps may be carried out in any desired order and two or more of thesteps may be carried out simultaneously with one another. Two or more ofthe steps disclosed herein may be combined in a single step, and in someembodiments, one or more of the steps may be carried out as two or moresub-steps. Further, other steps or sub-steps may be carried in additionto, or as substitutes to one or more of the steps disclosed herein.

Although inventive concepts have been described with reference to theembodiments illustrated in the attached drawing figures, equivalents maybe employed and substitutions made herein without departing from thescope of the claims. Components illustrated and described herein aremerely examples of a system/device and components that may be used toimplement embodiments of the inventive concepts and may be replaced withother devices and components without departing from the scope of theclaims. Furthermore, any dimensions, degrees, and/or numerical rangesprovided herein are to be understood as non-limiting examples unlessotherwise specified in the claims.

What is claimed is:
 1. A system, comprising: a head worn displayconfigured to display symbology of an operating environment; a trackingcamera; a tracking processing unit communicatively coupled to thetracking camera configured to: determine a position and orientation ofthe head worn display, transmit a position and orientation signal to acontroller, wherein the position and orientation signal includes asignal quality value; a controller communicatively coupled to the headworn display and the tracking camera comprising: one or more processors;and a memory with instructions stored upon, that when executed by theone or more processors, cause the one or more processors to: display thesymbology on the head worn display; determine a quality-of-service valuebased on at least the signal quality value; modify at least one symbolof the symbology based on the quality-of-service value from a defaultsymbol to a degraded symbol if the quality-of-service value changes froma high-quality value to a low-quality value.
 2. The system of claim 1,wherein the instructions further cause the one or more processors to:receive a sensor signal from one or more sensors; and use the sensorsignal to determine the quality-of-service value.
 3. The system of claim1, wherein the quality-of-service value comprises an accuracycharacteristic.
 4. The system of claim 1, wherein the quality-of-servicevalue comprises a precision characteristic.
 5. The system of claim 1,wherein the quality-of-service value comprises a signal strengthcharacteristic.
 6. The system of claim 1, wherein the degraded symbol isconfigured as a removed symbol.
 7. The system of claim 1, wherein thedegraded symbol is configured with a different color than the defaultsymbol.
 8. The system of claim 1, wherein the degraded symbol isconfigured with a different line width than the default symbol.
 9. Thesystem of claim 1, wherein the degraded symbol is configured with adifferent line type than the default symbol.
 10. The system of claim 1,wherein the controller is communicatively coupled to a vision system.11. The system of claim 1, further comprising an inertial sensorcommunicatively coupled to at least one of the head worn display or thetracking processing unit.
 12. The system of claim 1, wherein the symbolrepresents a confidence line of a path.
 13. The system of claim 12,wherein the path is configured as a runway.
 14. A method for modifyingsymbology displayed on a head worn display (HWD) based on aquality-of-service value comprising: determining a position andorientation of the HWD; transmitting a position and orientation signalto a controller, wherein the position and orientation signal includes asignal quality value; determining a quality-of-service value based on atleast the signal quality value; displaying symbology on the HWD, whereinat least one symbol of the symbology is modified from a default form toa degraded form based on the quality-of-service value.
 15. The method ofclaim 14, further including: receiving a sensor signal from one or moresensors; and using the sensor signal to determine the quality-of-servicevalue.